Modeling and optimizing MapReduce programs
نویسندگان
چکیده
منابع مشابه
Modeling and optimizing MapReduce programs
MapReduce frameworks allow programmers to write distributed, dataparallel programs that operate on multisets. These frameworks offer considerable flexibility to support various kinds of programs and data. To understand the essence of the programming model better and to provide a rigorous foundation for optimizations, we present an abstract, functional model of MapReduce along with a number of c...
متن کاملOptimizing MapReduce for Multicore Architectures
MapReduce is a programming model for data-parallel programs originally intended for data centers. MapReduce simplifies parallel programming, hiding synchronization and task management. These properties make it a promising programming model for future processors with many cores, and existing MapReduce libraries such as Phoenix have demonstrated that applications written with MapReduce perform co...
متن کاملSolving Linear Programs in MapReduce
Most interesting discrete optimization problems are NP-hard, thus no efficient algorithm to find optimal solution to such problems is likely to exist. Linear programming plays a central role in design and analysis of many approximation algorithms. However, linear program instances in real-world applications grow enormously. In this thesis, we study the Awerbuch-Khandekar parallel algorithm for ...
متن کاملAutomatic Optimization for MapReduce Programs
The MapReduce distributed programming framework has become popular, despite evidence that current implementations are inefficient, requiring far more hardware than a traditional relational databases to complete similar tasks. MapReduce jobs are amenable to many traditional database query optimizations (B+Trees for selections, column-storestyle techniques for projections, etc), but existing syst...
متن کاملOptimizing Network Usage in MapReduce Scheduling
Recent works [8, 19] have opened up an important optimization space for MapReduce scheduling by proposing models where map and shuffle phases are allowed to overlap. We study the MapReduce scheduling problem in the job-level model proposed in [19], focusing on the question of whether optimizing network usage can lead to better system performance. To achieve this, we introduce a technique called...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Concurrency and Computation: Practice and Experience
سال: 2014
ISSN: 1532-0626
DOI: 10.1002/cpe.3333